Writing a Scripting Engine

Hopper/SquoQuo

Writing demos (and synching) w/o a scripting engine is coding-your-brains-out as you might know. Wouldn't it be fun writing the effects methods and then dragging parts around in a track-based editor? To do so, we'll need two parts:

1. The editor
2. A managing method in our demo (the main thing)

All my code is written in C++, I hope you get the message, even if you're not used to C++. Some special constructs will be explained in more detail.

The editor

You can either write the editor for yourself (but it's gonna be a lotta work) or make yourself friends with the SquoQuo Demo Editor which I wrote for usage with my own scripting engine.

In this editor you can create, delete, rename, etc. (i.e. manage) so-called parts, which are mainly rectangles placed on the screen. Each part is placed on a track and can be moved around in time.
Each part holds several method entries, i.e. you can assign predefined (in your engine) methods to the part, sort them, etc.
Each method got parameters (as you all know ;) which can be set in different ways, too.

After this confusing introduction let me explain the structure doing a little example:
Let's say we want a pixel move around (neat style that is).
Simply write two methods, one that clears the screen and one that draws a pixel at a given position w/ given color. Both methods are assigned to the part, i.e. every time the part is executed both methods are called with the actual parameters.

Main thing about scripting is, we want the whole stuff synched to the sound. To do so, you can define ticks in the editor, parts can be snapped to ticks, etc.

Please see our editor page for further infos about the editor.

Now let's get into detail...

The file

The editor writes a project file, which holds all parts (sorted by start time) and for each part all method names and the parameters. The editor package contains the complete file format and an example implementation of the dispatcher, which reads the project file and does the managing stuff, like calling methods, calculating parameters, etc.

Main loop

Your main loop gets really simple now, something like this:

while (!stopped) {
   startFrame();       // new frame begins, initialize timer routines
   dispatcher->execute(getTimer());
   flipScreens();
}

Now all the stuff is done in dispatcher->execute(), it checks which parts are active, sorts the active parts by track number and executes them one by one. To determine which part is active, simple comparisons must be done, the easy way is to check for every part if the start is before the actual time and the end is after the actual time. This can get very slow, if you got many parts.
So, what to do?
If we sort all parts by ascending start time, we must find the first active part. Following parts will start later (i.e. later or at the same time), so simply check all following parts until a part is found with start>actual time. No other parts can be active!
If we save the actually-found first active part number, we can use it in the next frame to quickly determine the first part again.
Check this piece of code to see what I mean:

void PartDispatcher::execute(float time)
{
   int i, j, k;
   int calledParts[16];
   int dummy;

   if (time>=totalTime)
      setStopFlag(1);

   // determine, which is the first actual part
   i=actualFirstPart;
   while ((parts[i]!=NULL)&&(parts[i]->position+parts[i]->length<=time))
      i++;
   actualFirstPart=i;

   k=0;                                 // determine, which parts must be executed
   for (i=actualFirstPart; (parts[i]!=NULL)&&(parts[i]->position<=time); i++)
      if (parts[i]->position+parts[i]->length>time)       // part must be executed
         calledParts[k++]=i;

   // sort the parts depending on their track
   for (i=k-1; i>0; i--)
      for (j=0; j< i; j++)
         if (parts[calledParts[j]]->track>parts[calledParts[j+1]]->track) {
            dummy=calledParts[j];
            calledParts[j]=calledParts[j+1];
            calledParts[j+1]=dummy;
         }

   for (i=0; i<k; i++)
      parts[calledParts[i]]->execute(time);
}

What does this code?

- First, if the actual time exceeds the total demo time, then stop it!

- Then, determine (dependent on actualFirstPart, which holds the first part of the last frame) the first active part, this is done by stepping through the (sorted, remember that) parts, starting with the "old" first part, until we find a part which is still active (i.e. start+length>time).

- For all following parts which could be active (i.e. starttime) and save them in an array. We'll need this array to sort the active parts afterwards.

- Sort all active parts by ascending track number. Doing so, you can place the parts in the editor in a intuitive way:
All active parts are executed from top to bottom. Otherwise strange things can happen (believe me, I know it!) Bubble-sort works fine here due to rather small number of parts simultaneously active.

- Finally all active, now sorted parts can be executed one by one.

Executing a part

To execute a single part, there has to be some time calculations. Additionally, we've gotta calculate the parameters.
But first things first, here are the fields of a part:

char name[128];       // name of the part
float position;       // start time of part
float start;          // internal start time of part (added to actual time)
                      // *** use this for backwards playing
float length;         // length of part (in seconds)
float speed;          // 1.0=normal, -1.0=backwards, etc. (this does not apply to the
                      // parameters! only the time parameter is affected!)
char track;           // track number (used for sorting simultaneous parts)
Function **functions; // list of the functions that are executed by this part

position/length/track is explained above. They give the "position" in the time/track graph.
start/speed give some values used for the time calculation, i.e. the parts time will be (time-position), i.e. all parts start internally at time 0. Now speed is multiplied, resulting in (time-position)*speed. But what to do, if you want to stutter around in one part, say play 5 seconds, then repeat 1 second 3 times and then move on for 10 seconds?
A simple table should make things clear:

Part  position  length  start  speed
1     0.0       5.0     0.0    1.0
2     5.0       1.0     5.0    1.0
3     6.0       1.0     5.0    1.0
4     7.0       1.0     5.0    1.0
5     8.0       10.0    5.0    1.0

start is equal for parts 2-4, they only differ in position, e.g. at time 6.3 part 3 is executed. It's internal time is 0.3 but start is added, so all methods are called with time 5.3
You can use the start in multiple other ways, say playing parts backwards, etc.

At last, functions is an array of my class Function. This will be explained later. All non-C++ers should think of it as a simple array.
So, the part calls all its "functions" using the calculated time, which is (time-position)*speed+start. Here's the code:

void Part::execute(float time)
{
   if ((time<position)||(time>=position+length))	      // exit, if part isn't active
      return;

   for (int i=0; functions[i]!=NULL; i++)
      functions[i]->execute((time-position)*speed+start, (time-position)/length);
}

Well, not much done here, if the parts isn't active in fact, simply exit. Because this shouldn't happen, you could give an error message, too.
The second time value is used for the dynamic parameters, as they are all defined in range [0..1]. So this value is a time value running from 0.0 to 1.0

The class Function

First let me say, that Function is not what you may think, it's a wrapper which calculates the parameters and calls the engine methods. To do so, we need a special C++ construct, that is the function pointer. A function pointer is, as you might guess, a pointer to a function. Simply set the pointer and the function can be called in the usual manner. Advantage is, that we don't need any oversized case statement to call the functions.
You might wonder why I'm talking about functions instead of methods now. Unfortunately C++ can't handle pointers to methods, so all engine methods must be declared static, so in fact, they're functions, not methods anymore. To access class fields a this-pointer is given. Bit complicated, but it seems to be the only working way. If you know an easier way to do this, feel free to mail me!!!!
Let's take a look at the fields in class Function:

char name[128];                                // name of the part
void (*f)(void *tthis, float time, void **p);  // the function pointer itself
void *tthis;                                   // myown this-pointer used for the static
                                               // member functions
Parameter **parameters;

name should be obvious, tthis is the this pointer (see above).
parameters is an array (again) of my class Parameter, will be explained later...
The second line specifies the function pointer. It's a pointer to a function f(void*, float, void**). void* is the this pointer, this can be used (as said) to access class members. time is the part's internal time (see above) and the last void** is an array of void*, this is needed to support different parameter types, like strings, floats, etc. I'm not quite sure, how to do this in Pascal, but I guess, you'll find a way... Sorry for my lack of knowledge ;)

All the fields must be filled before the demo starts, this should be done, when reading in the project file. See the source of PartDispatcher:

void Function::execute(float time, float pTime)
{
   if (f==NULL)
      errorExit("No function call defined for %s!", name);

   void **params=(void**)calloc(9, sizeof(void*));   // allocate temporary parameters
                                                     // array
   for (int i=0; parameters[i]!=NULL; i++)           // fill parameters array
      params[i]=parameters[i]->getValue(pTime);
   f(tthis, time, params);                           // execute function
   free(params);                                     // and free the array
}

So, this is the last stage of managing things around ;)
We give an error message, if the function pointer is null. This should be impossible, but you'll never know...
Then, all parameters are parsed and the values are written to the array. We'll take a closer look at Parameter->getValue() soon. We call the function (by using the function pointer) and free the array afterwards. Nothing magical here I hope...

Parameter stuff

The Parameter class holds some fields, e.g. constant value, sine period, x/y coordinates, etc. For full description see the source codes.
What getValue() does, is it checks the type of the parameter (set when parsing the project file) and returns the corresponding value:

void *Parameter::getValue(float t)
{
   switch(type)
   {
      case PT_CONST:    temp=pConst;
                        break;
      case PT_SPLINE:   temp=calcSpline(x, y, dotCount, t);
                        break;
      case PT_STRING:   return pString;
                        break;
      default:          errorExit("Unknown parameter type\n");
   }
   
   return &temp;
}

This is only a brief version of the original source, but it shows the way. I use a static float value, named temp, to calculate float values, like constants, splines, sine. These values are calculated and afterwards the pointer to temp is returned.
Strings can be returned directly, as they're pointers to characters (in C++).

Now after all this managing stuff we should look at a little example of an engine method

The methods

To see, how the parameter handling and the "tthis" things work, I'll show you a little "hello world" engine method:

void SceneTest::helloWorld(void *tthis, float time, void **p)
{
   char *s=((char*)p[0]);
   float x=*((float*)p[1]);
   int i=*((float*)p[2]);
   
   warning("%s float %.2f int %d\n", s, x, i);
}

Now, what happens here?
p is an array of void*, i.e. we must cast the entries to the type we expect. So, the first entry is casted to char*, i.e. we read a string here. The second parameter is casted to float*, so it's a pointer to a float value, when dereferenced, we got the float value in x. The last parameter is a float value too, but finally casted to int.

The signature of all engine methods must be the same, otherwise the function pointer cannot be assigned! So, even if you don't need any parameters, the method must accept the third parameter void**!

Summing up

- The main loop calls all active Parts
- Every active part calls all its Functions
- All functions calculate all parameter values
- The engine method is called with the parameters

Some pitfalls should be named:

- Determining the active parts is a little tricky. I got some strange effects because I stopped at the first exceeded part. BUT there may be parts, that start later and end after the actual time (i.e. they're active)

- Sorting by ascending track number is essential, because otherwise the parts will be executed by ascending start time, which isn't what you want. Try it and you'll see what I mean

- It may be difficult to adapt this to Pascal or other languages, I don't know

- The function getTimer() (remember the PartDispatcher) MUST get the time value from the sound player, otherwise all synchronisation is - erh, let's say - gone. Check the documentation of the sound player on how to receive the actual time.

Granularity of the engine methods

Well, how many engine methods should one write? As they're the main methods of your demo (doing all the calculation) you should pay a few moments about this topic.
Personally I prefer larger methods, which got many parameters to keep things tight. For example I got animation methods, which get camera number, light intensity, animation values and other things and then, set up the whole 3d scene and render it.
But you could define methods for all state switches, too, say a method which wraps glBlendFunc() for example.

So, now my advise is, go play with the editor around and then read the source examples carefully.

Hopper/SquoQuo